Communication requires having a common language, a lingua franca, between agents. This language could emerge via a consensus process, but it may require many generations of trial and error. Alternatively, the lingua franca can be given by the environment, where agents ground their language in representations of the observed world. We demonstrate a simple way to ground language in learned representations, which facilitates decentralized multi-agent communication and coordination. We find that a standard representation learning algorithm -- autoencoding -- is sufficient for arriving at a grounded common language. When agents broadcast these representations, they learn to understand and respond to each other's utterances and achieve surprisingly strong task performance across a variety of multi-agent communication environments.

Large language models (LLMs) have demonstrated remarkable performance, particularly in multilingual contexts. While recent studies suggest that LLMs can transfer skills learned in one language to others, the internal mechanisms behind this ability remain unclear. We observed that the neuron activation patterns of LLMs exhibit similarities when processing the same language, revealing the existence and location of key linguistic regions. Additionally, we found that neuron activation patterns are similar when processing sentences with the same semantic meaning in different languages. This indicates that LLMs map semantically identical inputs from different languages into a "Lingua Franca", a common semantic latent space that allows for consistent processing across languages. This semantic alignment becomes more pronounced with training and increased model size, resulting in a more language-agnostic activation pattern. Moreover, we found that key linguistic neurons are concentrated in the first and last layers of LLMs, becoming denser in the first layers as training progresses. Experiments on BLOOM and LLaMA2 support these findings, highlighting the structural evolution of multilingual LLMs during training and scaling up. This paper provides insights into the internal workings of LLMs, offering a foundation for future improvements in their cross-lingual capabilities.

Communication requires having a common language, a lingua franca, between agents. This language could emerge via a consensus process, but it may require many generations of trial and error. Alternatively, the lingua franca can be given by the environment, where agents ground their language in representations of the observed world. We demonstrate a simple way to ground language in learned representations, which facilitates decentralized multi-agent communication and coordination. We find that a standard representation learning algorithm -- autoencoding -- is sufficient for arriving at a grounded common language.

AI-generated synthetic data allows to distill the general patterns of existing data, that can then be shared safely as granular-level representative, yet novel data samples within the original semantics. In this work we explore approaches of incorporating domain expertise into the data synthesis, to have the statistical properties as well as pre-existing domain knowledge of rules be represented. The resulting synthetic data generator, that can be probed for any number of new samples, can then serve as a common source of intelligence, as a lingua franca of learning, consumable by humans and machines alike. We demonstrate the concept for a publicly available data set, and evaluate its benefits via descriptive analysis as well as a downstream ML model.

Let's face it - the meteoric rise in digital and text communication has drastically changed the way we speak to one another. This ever-evolving shift in language creates a massive burden for ediscovery teams, who need to understand how text is used in context in order to effectively use legal technology to navigate massive amounts of data. In this episode, Amanda Jones of Lighthouse joins Bill and Rob to illuminate some common challenges and pitfalls that can arise with modern language in ediscovery. Let's face it - the meteoric rise in digital and text communication has drastically changed the way we speak to one another. This ever-evolving shift in language creates a massive burden for ediscovery teams, who need to understand how text is used in context in order to effectively use legal technology to navigate massive amounts of data.

Despite the surprising power of many modern AI systems that often learn their own representations, there is significant discontent about their inscrutability and the attendant problems in their ability to interact with humans. While alternatives such as neuro-symbolic approaches have been proposed, there is a lack of consensus on what they are about. There are often two independent motivations (i) symbols as a lingua franca for human-AI interaction and (ii) symbols as (system-produced) abstractions use in its internal reasoning. The jury is still out on whether AI systems will need to use symbols in their internal reasoning to achieve general intelligence capabilities. Whatever the answer there is, the need for (human-understandable) symbols in human-AI interaction seems quite compelling. Symbols, like emotions, may well not be sine qua non for intelligence per se, but they will be crucial for AI systems to interact with us humans--as we can neither turn off our emotions nor get by without our symbols. In particular, in many human-designed domains, humans would be interested in providing explicit (symbolic) knowledge and advice--and expect machine explanations in kind. This alone requires AI systems to at least do their I/O in symbolic terms. In this blue sky paper, we argue this point of view, and discuss research directions that need to be pursued to allow for this type of human-AI interaction.

Or in developed nations that are less wealthy than their closest neighbors, like my native Portugal. Because of the country's modest economic size, compared to most of Western Europe, many online companies have limited (or no) presence in Portuguese. British Airways, for instance, only offers customer service in Portuguese on weekdays during business hours--and they're a global airline with enormous operations in Europe. What's more, there are almost 230 million native speakers of Portuguese worldwide, the vast majority of them in Brazil (where, yes, British Airways also offers flights). It's the sixth most spoken language in the world.

This is a simple and straightforward guide to designing human-centered AI. The techniques mentioned herein were honed from our own experiences designing AI systems across industries and segments, for both consumers and enterprises. Our work in AI has taken us from finance to healthcare, from VR to photo-sharing, from asset management to predictive operations, and much more. Designing an AI to work within the messiness of the real world requires new frameworks, as novel challenges emerge within such dynamic and complex systems. While we cannot offer a step-by-step process to guaranteed innovation (no innovation can come without experience and tinkering), this guide attempts to distill our own learnings into a re-usable methodology.

I bought a train ticket online from Virgin Trains recently, to get me to St. Neots, nearest station to the site of this conference, where I'm speaking to an association of freelance editors. The follow-up email from Virgin Trains surprised me. The subject line said: "Your St. Neots journey, your way." Your St. Neots journey is a well-formed English noun phrase using the proper name St. Neots as an attributive modifier of the noun journey. Trivial to program: They simply had to take Your _____ journey and substitute into the blank the name of the destination I had picked.

The BBC is launching 11 new language services and one of them is English-based Pidgin, which is one of the most widely spoken languages across West Africa, even though it is not officially recognised. The Oxford English Dictionary definition of Pidgin is: A language containing lexical and other features from two or more languages, characteristically with simplified grammar and a smaller vocabulary than the languages from which it is derived, used for communication between people not having a common language; a lingua franca. Simply put, Pidgin English is a mixture of English and local languages which enables people who do not share a common language to communicate. Most African countries are made up of numerous different ethnic groups who do not necessarily have a lingua franca, so Pidgin has developed. It is widely spoken in Nigeria, Ghana, Equatorial Guinea and Cameroon.